Skip to content

fix(aio): validate skill reference links resolve in the built bundle - #89629

Merged
trunk-io[bot] merged 3 commits into
masterfrom
posthog-self-driving/fixai_observability-repair-missing-08abaa
Aug 27, 2026
Merged

fix(aio): validate skill reference links resolve in the built bundle#89629
trunk-io[bot] merged 3 commits into
masterfrom
posthog-self-driving/fixai_observability-repair-missing-08abaa

Conversation

@posthog

@posthog posthog Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Problem

  • An AI observability skill links a reference file that its bundle never ships, so an agent following the link during a required step dead-ends.
  • The skill build strips the .j2 suffix when it renders a template. A SKILL.md link to a .md.j2 path then points at a file the bundle does not contain.
  • exploring-llm-traces linked references/example-llm-trace.md.j2, but the bundle holds only the rendered references/example-llm-trace.md.
  • Nothing checked these links, so the break shipped without warning.

Note

The inbox report named creating-online-evaluationsreferences/evaluation-payload.md. That link is correct: the source holds evaluation-payload.md.j2, which the build renders to evaluation-payload.md, so the bundled link resolves. The real break is the reverse case in exploring-llm-traces, which the new check now catches.

Changes

  • The skill lint now fails when a SKILL.md links a references/ or scripts/ path that the built bundle will not contain.
  • exploring-llm-traces now links the rendered references/example-llm-trace.md, so the reference opens.
  • Mechanical: the check resolves each link against the shipped file set and accepts a .md link when the source holds either the .md file or its .md.j2 template. It scans only the SKILL.md entry point.

How did you test this code?

  • Added two lint cases in test_build_skills.py: a SKILL.md linking a .md.j2 template fails the lint; a SKILL.md linking the rendered .md of a template passes. These catch a reintroduced template-path link, which no existing test covered.
  • Ran the skill lint over the whole repo. It passes after the link fix and flags nothing else.
  • Not run: the database-backed suites; this change is lint-only.

Automatic notifications

  • Publish to changelog?

Docs update

None.

🤖 Agent context

Autonomy: Fully autonomous

  • Authored by an agent (Claude Code) from an inbox report about a dead-ending skill reference link.
  • Skills invoked: /writing-tests, /writing-pr-descriptions, /writing-simplified-technical-english.
  • The report proposed either repointing the link to .md.j2 or adding a static .md. Both are wrong here: repointing to .md.j2 breaks the bundle, and a static .md would drift from the Pydantic-rendered template. The durable fix is the publication check the report also suggested, which confirmed the named skill is fine and surfaced the genuine break in a sibling skill.

Created with PostHog Desktop from this inbox report.

The skill build strips the .j2 suffix from every rendered template, so a
SKILL.md that links to a `.md.j2` reference points at a path the bundle never
contains. Add a blocking lint check that resolves every references/ and scripts/
link in each SKILL.md against the shipped bundle, and fix the one skill that
linked a template path.

Generated-By: PostHog Desktop
Task-Id: 51745b3a-bba0-431b-8b69-fc0e4170d7c2
@posthog

posthog Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

🦔 ReviewHog reviewed this pull request

Nothing worth raising this time, so here's a calming picture instead:

A panda relaxing and waving

@trunk-io

trunk-io Bot commented Aug 26, 2026

Copy link
Copy Markdown

😎 Merged successfully - details.

@github-actions

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

@trunk-io

trunk-io Bot commented Aug 26, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@carlos-marchal-ph
carlos-marchal-ph self-requested a review August 27, 2026 09:43
@carlos-marchal-ph
carlos-marchal-ph marked this pull request as ready for review August 27, 2026 09:55
@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team August 27, 2026 09:55
@pr-assigner-resolver-posthog

Copy link
Copy Markdown

👀 Auto-assigned reviewers

These soft owners were skipped because they only have minor changes here. Nothing blocks merge, so self-assign if you'd like a look:

  • @PostHog/team-ai-observability (products/ai_observability/product.yaml)

Soft owners come from each directory's owners.yaml and each product's product.yaml (resolved nearest-file-wins). The locator after each owner is the file that decided it. Generated files and lockfiles are ignored when deciding ownership.

@carlos-marchal-ph

Copy link
Copy Markdown
Contributor

/trunk merge

@trunk-io
trunk-io Bot merged commit d0855e5 into master Aug 27, 2026
292 of 345 checks passed
@trunk-io
trunk-io Bot deleted the posthog-self-driving/fixai_observability-repair-missing-08abaa branch August 27, 2026 12:54
@deployment-status-posthog

deployment-status-posthog Bot commented Aug 27, 2026

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-08-27 14:12 UTC Run
prod-us ✅ Deployed 2026-08-27 14:31 UTC Run
prod-eu ✅ Deployed 2026-08-28 07:22 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant